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It  is  ,3r£ued  that  the  clinical  and  statistical  approaches  rest  on 
different  assumptions  about  the  nature  of  random  error  and  the  appropriate 
level  of  accuracy  to  be  expected  in  prediction.  To  examine  this,  a  case  is 
made  for  each  approach.  The  clinical  approach  is  characterized  as  being  .  . 
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Accsf  tiftf  Emr  te  Ibka  Lm  Emr 

Hiltol  J.EiaAora 
C»at«r  for  Dscinoa  BMMrch 
UaireniCf  of  Chictf  o 

Qa  a  rectal  profraa  of  "Yall  Street  Veek."  the  eaiaeat  ecoaomist,  lliltoa  Friedaaa, 
wu  beiai  iateryieered  by  Louis  Rukeyser.  Mr.  Rukeyser,  Tho  is  e  cyaic  about  cliaical 
aad  statistical  predictioa,  asked  Friedaiaa  vhat  he  yould  do  about  the  Federal  Reserve 
Board,  erhich  has  beta  aa  object  of  Friedsua's  criticise  for  laaay  years.  Without 
ssissiag  a  beat.  Friedsua  replied,  “I  erould  get  rid  of  thea."  Alter  expressiag  surpriae. 
Rukeyser  asked.  "What  vould  you  use  to  replace  them  r  ‘A  computer,"  respoaded  Mr. 
Friedsua.  He  thea  treat  oa  to  explaia  that  the  auaey  suppy  should  be  set  by  usiag  a 
sissple  rule  that  is  coasisteatly  applted.  This  trould,  he  argued,  provide  for  more 
stability  aad  certaiaty  ia  detersdaiag  ecoaoaic  policy.  Whatever  the  sserits  of  his 
argusseat,  I  feel  coafideat  (i.e.,  probability  -  .999)  that  the  idea  of  replaciag  the  Federal 
Reserve  Board  by  a  computer  algorithss  trill  seem  absurd  aad  daagerous  to  ssost  people. 
Be  that  as  it  suy,  the  poiat  of  this  example  is  to  illustrate  that  the  cliaical  vs.  statistical 
predictioa  coatroversy  is  eaduriag  aad  geaeral.  Not  oaly  is  the  coatroversy  alive  aad 
▼ell  ia  ecoaoaiics,  I  believe  that  the  rapid  grotrth  ia  computer  use  trill  spread  the 
coafUct  to  aetr  fields  aad  iateasiiy  the  battle  trhere  it  already  exists. 

The  purpose  of  this  paper  is  to  uaderstaad  wby  the  coatroversy  exists  aad  persists, 
la  Thai  foUotrs,  I  argue  that  the  cliaical  aad  statistical  approaches  rest  oa  quite 
dilTereat  philoeophical  assumptioas  about  the  aature  of  error  aad  the  appropriate 
level  of  accuracy  to  be  expected  ia  predictioa.  To  exaaiiae  these  issues,  a  case  is  aiade 
for  each  approach.  Thereafter,  a  decisioa  aaalysis  is  iatroduced  to  examiae  the  costs 
aad  beaefits  of  subscribiag  to  each  positioa. 

The  cliaical  approach 

la  their  diagaostic  activities,  cliaiciaasare  deieradaists.  That  is,  symptoms,  sigas, 
aad  the  like,  are  viewed  as  maaifestatioas  of  uaderlyiag  causal  processes  that  caa  be 


knoTii  iA  principle.  Since  aach  clinical  reaeoninf  inTolres  diafnoais  or  backward 
inference  (ie..  aakinf  inferences  froa  effects  to  prior  causes),  the  clinican,  like  the 
hiaorian.  has  auch  latitude  (or  dofrees  of  freedoa)  in  reconstructing  the  paa  to  suke 
the  present  soea  moA  likely  (a  kind  of  suiiaua  likelihood  approach,  if  you  will). 
However,  when  engaging  in  predictton  or  foward  inference,  one  is  soon  confronted 
with  discrepancies  between  predicted  and  actual  outcoaes.  Such  discrepancies  are 
often  surprising,  especially  if  the  explanation  for  paa  behavior  provides  a  coherent 
account  of  the  facts.  One  is  reminded  of  the  unpleasant  surprise  that  awaits  the  modeler 
who  fits  the  data  with  aany  paraaears  only  a  find  that  the  model  cannot  predict  new 
cases.  Thus,  it  is  olUa  the  case  that  the  power  of  poa  hoc  explanation  is  aatched  by  the 
paucity  of  predictive  validity. 

Given  the  fluency  of  causal  reasoning,  it  is  not  difficult  a  construct  reasons  for 
why  discrepancies  in  prediction  occur.  Indeed,  in  hindsight,  it  seesss  as  if  the  ouaoaes 
could  not  have  been  otherwise  (see.  Fischhoff,  1973  on  ‘hindsight  bias‘  as  a  fora  of 
creeping  dearainisa).  However,  a  wha  degree  can  (should)  prediction  errors  be 
explained  ?  It  is  a  this  point  tha  the  clinica  and  stttistica  approaches  diverge,  with 
the  divergence  having  much  a  do  with  the  meaning  and  significance  of  ‘random 
error.‘  White  the  concept  of  randossnea  is  complex  and  difficult  a  define  (Lopes, 
1962),  it  suggesa  an  irreducible  unpredictability  and  disorder  of  ouaoaes.  The  basic 
question  then  becomes,  how  auch  of  behavior  is  random  and  how  auch  sysasutic  ? 
The  answer  a  this  depends  on  what  is  meant  by  randonmess.  For  example,  consider  the 
random  walk  theory  of  sack  market  prices.  While  most  people  have  not  heard  of  this 
theory,  amy  have  first  hand  experience  with  its  implications.  Do  sack  prices  follow  a 
random  walk  ?  To  date,  the  surket  remains  difficult  a  predict  (suny  have 
unsuccessfully  tried).  However,  does  that  mean  that  it  is  impossible  a  do  a  7  tesagine 
that  there  is  a  7-way  interaction  that  predicts  price  changes,  but  no  one  has  yet 
induced  it  from  the  mass  of  complex  and  noisy  data  that  is  available.  If  there  is  hidden 
sytematicity,  one's  gamble  in  searching  for  a  predictive  rule  may  pay  off.  On  the  other 


htAd,  such  ta  intenctiott  may  not  exist,  despite  the  fact  that  there  are  "experts  " 

selling  adrice  on  xrhat  stocks  to  buy  (  are  they  simply  selling  "snake  oil*?).  Thus,  if 

prediction  error  is  due  to  our  lack  of  knowledge  and  randomness  is  only  a  label  for  our 

current  ignorance,  there  are  at  least  two  reactions.  The  first  is  characteristic  of  the 

clinical  approach:  it  says  that  the  goal  of  perfect  predictability,  while  difficult  to  attain. 

is  not  impossible  Moreover,  this  goal  may  be  useful  in  itself  since  it  can  motivate  the 

search  for  improved  predictability  via  increased  understanding  of  the  causal  texture  of 

the  environment  (  Tolman  &  Brunswik.  1939)  The  second  reaction  is  characteristic  of 

the  statistical  approach  and  emphasizes  the  possibility  of  a  futile  search  for  a  Holy 

Grail.  This  is  considered  in  greater  detail  below 

The  importance  of  causal  understanding,  which  is  essential  to  the  clinical  approach, 

has  other  implications  While  the  controversy  between  the  clinical  and  statistical 

approaches  centers  on  prediction,  has  there  been  too  much  attention  given  to 

prediction  ?To  illustrate,  consider  the  following  scenario; 

Imagine  that  you  lived  several  thousand  years  ago  and  belonged  to  a 
tribe  of  methodologically  sophisticated  cave-dwellers  Your 
methodological  sophistication  is  such  that  you  have  available  to  you 
all  present  day  means  of  the  methodological  arsenal  -  details  of  the 
principles  of  deductive  logic,  probability  theory,  access  to 
computational  equipment,  etc.  However,  your  level  of  substantive 
knowledge  lags  several  thousand  years  behind  your  methodological 
sophistication.  In  particular,  you  have  little  knowlege  about  physics, 
chemistry  or  biology.  In  recent  years,  your  tribe  has  noted  an 
alarming  decrease  in  its  birth-rate.  Furthermore,  the  tribe's 
statistician  estimates  that  unless  the  trend  is  shortly  reversed, 
extinction  is  a  real  possibility  The  tribe's  chief  has  accordingly 
launched  an  urgent  project  to  determine  the  cause  of  birth .  You  are  a 
member  of  the  project  team  and  have  been  assured  that  all  means, 
ittclhding  various  forms  of  experimentation  with  human  subjects, 
will  be  persiitted  to  resolve  this  crucial  issue 
(Einhom  h  Hogarth,  1982,  p23) 

The  above  story  illustrates  the  following  points:  ( 1 )  The  goal  of  prediction  is  to  provide 
guidance  for  taking  action  Therefore,  prediction  is  intimately  tied  to  decision  making 
and  should  be  evaluated  within  this  context  Indeed,  one  might  find  small  consolation 
in  being  able  to  accurately  predict  when  the  tribe  will  become  extinct:  (2)  When 
decisions  are  based  on  predictions,  the  determination  of  forecast  accuracy  is 
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probleAttic  since  outcomes  are  a  function  of  predictions  and  actions  (Einhom  & 

Hogarth.  197S).  For  eiample,  if  the  President  takes  strong  anti-recession  measures 

based  on  predictions  of  an  economic  slovdovn.  hov  is  one  to  evaluate  the  accuracy  of 

the  forecast  ?  Consider  the  outcome  of  “no  recession  “  This  could  result  from  an 

incorrect  forecast  and  a  useless  action,  gc.  an  accurate  forecast  and  a  highly  effective 

action.  Similarly,  a  recession  could  indicate  an  accurate  forecast  vith  an  ineffective 

action,  or.  an  inaccurate  forecast  vrith  an  action  that  causes  the  malady  it  is  intended  to 

prevent.  While  some  actions  are  taken  to  counteract  the  prediction  of  undesirable 

events,  other  actions  cause  the  very  outcomes  that  are  predicted.  For  eiample. 

People  in  a  small  tovn  hear  a  rumor  that  the  banks  are  about  to  fail. 

They  think  that  if  this  forecast  is  accurate,  they  had  better  srithdrav 
their  money  as  soon  as  possible.  Accordingly,  they  go  to  the  banks  to 
close  their  accounts  (those  sceptical  of  the  forecast  see  many  people 
vithdravring  money  and  either  take  this  as  a  sign  that  the  rumor  is 
true  or  foresee  the  consequences  of  waiting  too  long,  thus  joining 
the  crovrd  in  either  case).  By  the  end  of  the  day  the  banks  have 
failed,  thereby  confirming  the  rumor.  (Einhom  U  Hogarth.  1^. 
p.24) 

Note  that  awareness  of  such  self-fulfilling  prophecies  is  often  low  and  can  lead  to 
overconfidence  in  predictions  that  are  of  low  or  even  zero  accuracy  (see  Einhom  & 
Hogarth.  1978;  Einhom.  1980);  (3)  In  order  to  understand  the  relations  between 
predictions,  actions,  and  outcomes,  one  needs  a  causal  model  of  the  process.  In  this 
regard,  the  clinical  approach,  with  its  emphasis  on  diagnosis  and  causal  understanding, 
is  important.  Moreover,  the  role  of  clinical  judgment  in  the  development  of  models  and 
the  deteraaination  of  relevant  variables  has  been  sadly  neglected.  Consider  the 
following  conclusion  from  the  literature  on  clinical  vs.  statistical  prediction  stated  by 
Dawes  and  Oirrigan  (1974.  p.lO?)-  “ .  the  whole  trick  is  to  decide  what  variables  to  look 
at  and  then  to  know  how  to  add.”  Assuming  we  can  add.  how  do  we  decide  on  what 
variables  to  look  at  ^  Such  decisions  must  rest  on  some  implicit  or  eiplicit  theory  of  the 
phenomenon  which  allows  one  to  distinguish  relevant  from  irrelevant  factors 
Therefore,  prediction  depends  on  backward  inference  which  involves  both  the 
forming  of  hypotheses  to  interpret  the  past  and  the  choosing  of  relevant  from 
irrelevant  variables  in  that  interpretation 


ftltlinut  tjM  cUnicsl  •pproach  rests  on  the  voithy  tad  optiibstic  goal  of  perfect 
predictahilitf.  it  is  a  goal  that  can  hare  negatire  consequences  (see  beloer).  The 
statistical  approach,  on  the  other  hand,  accepts  error.  This  acceptance  can  occur  in 
several  vays.  First,  one  may  believe  that  the  vorld  is  inherently  uncertain.  In  this 
case,  probabilistic  knowledge  is  the  best  tre  can  hope  for  and  random  error  cannot  be 
reduced  by  greater  knovrledge.  Second,  one  can  maintain  determinism  at  the  level  of 
the  physical  world  but  believe  that  our  knowledge  of  that  world  will  always  be 
fragsaentary  and  hence  uncertain.  In  this  case,  randosaness  is  due  to  ignorance  but  the 
goal  of  perfect  predictability  is  abandoned  as  being  too  unrealistic.  Third,  the  use  of 
any  equation  or  algorithm,  with  its  lisaited  number  of  variables  and  swchanical 
cosabining  rule,  can  never  capture  the  richness  and  fUU  compieiity  of  the 
phenosaenon  it  is  meant  to  predict  (recall  Meehl's  discussion  of  'broken  leg  cues',  1994. 
p.  29).  Thus,  models  are  simplifications  of  reality  that  must  lead  to  errors  in  prediction 
(cf.  Chapanis,  1961). 

Let  us  now  consider  how  the  acceptance  of  error  can  lead  to  less  error.  To  do  so. 
recall  the  research  on  probability  learning  done  several  years  ago  (e.g.,  Edwards.  1996, 
Estes,  1962).  In  these  studies,  either  a  red  or  green  light  is  illuminated  on  each  of  a 
number  of  trials  and  subjects  are  asked  to  predict  which  light  will  go  on.  If  the 
prediction  is  correct,  subjects  are  given  a  cash  payoff:  if  the  prediction  is  wrong,  there 
is  no  payoff.  However,  unbeknownst  to  the  subject,  the  lights  are  programmed  to  go  on 
according  to  a  binomial  process  with  a  given  proportion  of  red  and  green,  say  60%  red 
and  40%  green.  Thus,  the  process  is  random  although  subjects  do  not  know  this.  The 
major  result  of  these  eiperiments  is  something  called  'probability  matching';  i.e., 
subjects  respond  to  the  lights  in  the  same  proportion  as  they  occur.  For  eiample,  in  the 
above  case,  subjects  predict  red  60%  of  the  time  and  green  40%  .  The  eipected  payoff  for 
such  a  strategy  can  be  calculated  as  follows:  since  the  subject  predicts  red  on  60%  of  the 
trials  and  red  occurs  on  60%.  the  subject  will  be  correct  (and  receive  the  payoff)  on 


36%  of  tho  irials.  Siailtrly  for  groofi;  tho  subject  predicts  freeii  on  40%  of  the  triab 
tad  freon  occurs  on  40%.  Hence,  16%  of  the  trials  vill  be  correctly  predicted. 
Therefore,  oeer  both  predictions,  subjects  vill  be  correct  on  36%  «  16%  -  32%  of  the 
triab.  Nor  consider  hov  troll  subjectt  vould  do  by  usinf  a  sbapk  rub  that  said;  ahrays 
predict  the  most  lihety  color.  Note  that  such  a  strategy  accepb  error:  hoveyer,  it  also 
leads  to  60%  correct  predictions  ( i.e..  I  atvays  say  red  and  red  occurs  60%  of  the  time). 
Since  60%  bgreater  than  32%,  subjectt  trould  suite  more  money  if  they  accepted  error 
and  consistently  used  a  simple  rule.  Indeed,  such  a  rule  their  wealth  in  thb 

situation.  Hoverer,  most  are  trying  to  predict  perfectly  and  are  engaged  in  a  futile 
attempt  to  see  patterns  in  the  data  that  are  diagnostic  of  the  (non-eiistent)  rub  that 
they  beliere  debrsaines  the  onset  of  the  lights.  (The  anabgy  to  the  stock  market  b 
noted  without  Djither  conunent.) 

Another  esample  of  accepting  error  to  make  less  error  comes  from  the  work  on 
equal  or  unit  weights  in  linear  modeb  (Dawes  U  (^rrigan,  1974:  Einhorn  &  Hogarth, 
1973).  Many  people  are  surprised  that  equally  weighted  linear  regression  modeb  can 
outpredict  modeb  with  ’’optioul"  weights,  on  new  cases.  The  reason  for  the  surprise  b 
that  we  often  beliere  that  the  weights  for  the  rariables  are  not  equal.  Thus,  the  use  of 
equal  weights  deliberately  introduces  error  into  the  model.  However,  there  b  a  benefit 
from  such  a  procedure:  viz.,  equal  weighting  protects  one  against  a  reversal  of  the 
relative  weighting  of  the  variables  on  the  baab  of  poor  data.  Thus,  if  XI  and  12  have  a 
true  relative  weighting  of  say  2:1,  equal  weights  protect  one  from  data  which  shows 
that  the  waight  for  12  b  larger  than  for  II .  Therefore,  if  data  are  of  sufficiently  poor 
quality,  seeking  error  can  lead  to  toss  error  in  prediction. 

Yhito  the  idea  of  trade-offs  amongst  errors  may  be  new  to  some,  there  are  several 
more  mundane  advantages  of  the  statistical  approach  that  nonetheless  deserve 
euntion.  First,  the  statistical  approach  demands  that  empirical  evidence,  rather  than 
authority,  be  the  deciding  factor  in  detersiining  the  predictive  accuracy  of  any  device. 
Hence,  the  statistical  approach  b  egalittfian-  it  trusts  no  one  and  takes  little  on  faith 


In  fact.  Araatrong  s  (1978)  notion  of  a  “seersucker  theory  of  prediction*  seems  to 
captures  the  Mitude  of  the  statistical  approach  to  all  undocumented  claims  of  expertise 
The  theory  has  only  one  axiom:  for  every  seer  there  is  a  sucker  A  second  issue 
concerns  inconsistency  in  judgment  due  to  fatigue,  boredom,  memory  and  auentional 
limitations,  and  so  on  .  Such  inconsistency  is  not,  in  general,  useful  Indeed,  if  someone 
has  a  valid  rule  which  is  inconsistently  applied,  predictive  accuracy  will  suffer 
However,  clinicial  judgment  can  be  improved  by  techniques  such  as  "bootstrapping.* 
in  which  a  model  of  the  clinical  thought  processes  predicts  more  accurately  than  the 
person  from  whom  the  model  was  developed  (Goldberg.  1970).  Such  models  have  been 
developed  in  many  fields  and  the  results  are  encouraging  (see  Camerer,  1981  for  a 
review  and  theoretical  discussion) 

A  Decision  Analysis 

Since  I  have  tried  to  make  a  case  for  both  the  clinical  and  statistical  approaches, 
the  question  naturally  arises,  which  is  better  ?  Such  a  naive  question  deserves  an 
answer  like,  "it  depends "  This  section  considers  some  of  the  factors  upon  which  it 
depends. 

To  begin,  consider  Figure  1.  which  shows  a  decision  matrix  with  choices  as  rows  and 
states  of  the  world  as  columns  For  the  sake  of  simplicity,  only  two  choices  and  states  are 


Insert  Figure  1  about  here 


shown.  First  consider  the  choice  alternatives:  one  can  decide  that  a  phenomenon  of 
interest  is  systematic  and  thus  capable  of  being  predicted:  or.  one  can  decide  that  the 
phenomenon  is  random  and  not  predictable  Now  consider  the  possible  states  of  nature 
In  the  first  column,  the  phenomenon  is  systematic,  while  in  the  second  it  is  random 
The  intersection  of  rows  and  columns  results  in  four  possible  outcomes;  the  "hits '. 
shown  in  the  diagonal,  and  the  errors,  shown  in  the  off-diagonal  Note  that  there  are 


two  iinds  of  errors.  If  one  decides  thtt  n  phenomenon  is  systematic  and  it  is  random, 
the  error  that  result  is  manifested  in  myths,  magic,  superstitions,  and  illusions  of 
control  (Langar,  197?).  This  error  is  most  lilely  to  characterize  the  clinical  approach, 
▼hich  seeks  causal  explanations  for  all  behavior  Moreover,  there  are  numerous 
examples  of  this  type  of  error  which  have  been  discussed  in  the  behavioral  decision 
theory  literature  (for  a  review,  see  Einhorn  &  Hogarth.  19S1:  Nisbeu  &Ross.  1980) 

Let  us  now  consider  the  other  error,  which  is  more  likely  to  characterize  the 
statistical  approach  In  this  case,  one  decides  that  a  phenomenon  is  random  when  it  is 
systematic.  This  error  results  in  lost  opportunities  and  illusions  of  the  lack  of  control. 
For  example,  consider  the  state  of  knowledge  of  the  movement  of  heavenly  bodies  after 
Copernicus  but  before  Kepler.  The  Copernican  revolution  put  the  sun  at  the  center  of 
the  solar  system  with  the  planets  revolving  in  circular  orbits.  This  model  of  planetary 
motion  gives  reasonably  accurate  predictions  However,  we  know  that  the  orbits  are  not 
circular:  they  are  elliptical  and  errors  in  prediction  occurred.  If  probabilism  were 
around  in  the  time  of  Copernicus,  one  might  have  explained  planetary  motion  as 
consisting  of  circular  orbits  plus  a  random  error  term  While  such  a  probabilistic  model 
would  explain  most  of  the  variance,  it  would  represent  a  lost  opportunity  to  better 
understand  the  true  nature  of  the  phenomenon  Of  course,  successes  in  seeking  to 
explain  all  the  variance  of  behavior  are  dramatic  However,  dramatic  failures  also  exist. 
Recall  Einstein  s  famous  statement  that.  '  God  does  not  play  dice  with  the  world."  His 
unsuccessful!  attempts  to  disprove  quantum  theory  illustrate  the  difficulty  of 
abandoning  the  goal  of  perfect  predictability 

What  conclusions  can  be  drawn  from  the  above  analysis  ^  First,  the  choice  between 
the  clinical  and  statistical  approach  in  any  given  situation  will  depend  on  (a)  One's 
beliefs  regarding  the  probabilities  of  the  states  of  nature  While  I  have  only  considered 
two  states,  there  are  many  states  representing  various  levels  of  systematicity  and  error 
Hence,  one's  prior  probabilities  over  the  various  states  will  greatly  affect  the  choice  of 
strategy:  (b)  The  relative  costs  of  the  two  types  of  errors  For  example,  to  what  degree  is 
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superstition  sn  approprinte  price  to  pay  for  not  missing  an  opportunity  to  predict  more 
accurately  ?;  and  (c)  The  relatiye  payoffs  for  the  hits/correct  choices.  Hence,  the 
choice  betveen  the  clinical  and  statistical  approaches  can  be  seen  as  a  special  case  of 
decision  making  under  uncertainty;  each  has  its  associated  risks  and  potential  benefits. 
At  the  least,  this  conceptualization  demonstrates  trhy  the  controversy  ▼ill  never  be 
resolved.  Researchers  will  “place  their  bets“  differently,  vhether  the  field  be 
personality  theory  or  particle  physics 

Cnnclnsinn 

The  clinical  vs  statistical  controversy  represents  a  basic  conflict  about  the 
predictability  of  behavior  While  the  evidence  is  clear  and  convincing  that  the 
statistical  approach  does  a  better  job  of  forecasting,  the  clinical  approach  is  not  without 
its  virtues  Indeed.  I  tend  to  think  of  the  clinical  approach  as  a  high  risk  strategy  -  i.e., 
the  chance  of  being  able  to  predict  all  the  variance  of  behavior  (or  even  a  substantial 
amount),  is  very  low.  but  the  payoff  is  correspondingly  high.  On  the  other  hand,  the 
acceptance  of  error  to  make  less  error  is  likely  to  be  a  safer  and  more  accurate  strategy 
over  a  wide  range  of  practical  situations.  Thus,  the  statistical  approach  leads  to  better 
performance  on  average  In  my  view,  this  is  a  compelling  argument  for  its  use 
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